On Projective Rectification

We present a novel algorithm performing projective rectification which does not require explicit computation of the epipolar geometry, and specifically of the fundamental matrix. Instead of finding the epipoles and computing two homographies mapping the epipoles to infinity, as done in recent work on projective rectification, we exploit the fact that the fundamental matrix of a pair of rectified images has a particular, known form. This allows us to set up a minimization that yields the rectifying homographies directly from image correspondences. Experimental results show that our method works quite robustly even in the presence of noise, and can cope with inaccurate point correspondences. INTRODUCTION The cornerstone for stereo and motion 2-frame analysis is the solution of the correspondence problem which can be synthesized as determining which parts of two images, say and , are projections of the same scene element. The solution of this problem is given by a correspondence map, or equivalently a disparity map, which relates each pixel of to at most one pixel in . In order to determine the correspondence map for each pixel in a pixel maximizing a similarity measure must be determined (Ayache [1]). The geometry of a stereo system, called epipolar geometry and codified in the fundamental matrix (Zhang [17]), is such that correspondent points are constrained to lie on particular lines called epipolar lines (Faugeras [3]). Therefore if the fundamental matrix is known the correspondence problem is reduced from a 2D search to a 1D search problem. Needless to say, the search for a correspondent point can be simplified if the two images are warped (Wolberg [16]) in such a way that the correspondent points lie on the same scan-line in the two images. In other words, the epipolar lines are parallel to the horizontal image axes. Note that most of the stereo algorithms presented in the literature assume this configuration. This process is called rectification, and the two transformed images can be regarded as obtained by a stereo rig obtained rotating the two original cameras. The concept of rectification has been known for long to photogrammetrists [14], and their approach was merely optical. Most of the approach by vision researchers, like the ones presented by Ayache and Hansen [2], and Fusiello et al [4], assume the two projection matrices are known. Only recently algorithms not assuming a full calibration of the two cameras, but only a weak calibration (knowledge of the epipolar geometry) have been presented by Robert et al. [11], Seitz and Dyer [12], Papadimitriou and Dennis [9]. In particular, Hartley [7] gives a theoretical presentation of uncalibrated rectification. All these algorithms rely on the estimation of the fundamental matrix . Despite the fact that several methods for computing the fundamental matrix have been published by Luong and Faugeras [8] and [17], no algorithm has proved to be stable in every situation [8], making this a still open topic of research. In this paper we present a novel algorithm performing projective rectification which does not require explicit computation of the epipolar geometry, and specifically of the fundamental matrix. We exploit the fact that the fundamental matrix of a pair of rectified images has a particular, known form to set up a minimization yielding the rectifying homographies directly from image correspondences. The two transformations computed by our algorithm can then be used to estimate the epipolar geometry between the two original images. Our algorithm makes use of theoretical results presented in [7]. In the following subsection our notation is given. In the second section the epipolar geometry of a rectified stereo pair is characterized. Some results on the rectifying transformations are reviewed in the third section. The rectification algorithms is presented in the fourth section and experimental results are given in the fifth section. The last section is dedicated to final remarks and a brief discussion. Notation It is convenient to cast our presentation from the point of view projective geometry [13], whereby the image planes are considered as projective planes, and image points are represented as 3D column vectors. A rectifying transformation is a linear one to one transformation of the projective plane, called homography, represented by a non-singular matrix. We indicate column vectors by bold lower-case letters, such as . Row vectors are denoted by transposed column vectors, e.g., . Matrices are denoted by bold upper-case letters, e.g., . Scalars are denoted by italic letters. Given a vector we denote by the rank-2 skew-symmetric matrix used in place of the vector prod-